Perceptual study

Introduction

We conducted a perceptual study with 32 participants (aged 19 to 32) to further assess the effectiveness of our method. We provide an interative GUI similar to 2-2 all layers.html for participants to evaluate the vectorization results generated by Photo2ClipArt and our method. Specially, for each of the 32 examples, all participants were shown the input image and two anonymous vectorization results generated by Photo2ClipArt and our method, respectively, and these two anonymous results are arranged in random order.

The visualization of vectorization results showed all semi-transparent layers and allowed participants to interactively choose subsets of the layers to composite. For each example, participants were asked three questions:
Q1 (reconstruction quality): Which result do you think is closer to the input image?
Q2 (shape consistency): Which decomposition better reflects the parts of the input shape?
Q3 (editing convenience): Which decomposition is more convenient for editing?
Participants could choose a vectorization result or indicate that they were equivalent.

For Q1, Q2, and Q3, our method received more votes in 27 (84%), 30 (94%) and 30 (94%) of the 32 examples, respectively. We also performed a chi-squared test on each voting result of our user study and consistently produced a p-value << 0.0001. The perceptual study results show that our approach generates consistently superior results than Photo2ClipArt. Detailed voting results are shown the bellow tables.

Detailed voting results for each question

Table 1. Voting results of "Q1 (reconstruction quality): Which result do you think is closer to the input image?"
Id Example Poll of Ours Poll of Photo2ClipArt Poll of Equivalent
1 Can 31 0 1
2 Cone 29 1 2
3 Purple-circle 16 11 5
4 Hammer 13 10 9
5 Tiger 11 10 11
6 Cherry 10 15 7
7 Torch 8 16 8
8 Rocket 18 4 10
9 Plane 7 5 20
10 Soda 4 19 9
11 Trees 23 3 6
12 Teapot1 11 5 16
13 Syn0 20 3 9
14 Syn2 4 7 21
15 Syn4 25 3 4
16 Battery 24 2 6
17 Car 18 2 12
18 Coffee 20 8 4
19 Cow 23 4 5
20 Truck 11 6 15
21 House 15 2 15
22 Sound 23 3 6
23 Bear 10 8 14
24 Mouse 22 6 4
25 Lamp 18 5 9
26 Teapot2 17 3 12
27 Shoe1 13 7 12
28 Syn6 25 2 5
29 Syn8 17 3 12
30 Syn1 14 5 13
31 Syn6 17 3 12
32 Syn7 25 0 7

Average Voting Rate

542 (52.9%) 181 (17.7%) 301 (29.4%)
Conclusion In 27 examples (84%), our results get more votes than Photo2ClipArt.


Table 2. Voting results of "Q2 (shape consistency): Which decomposition better reflects the parts of the input shape?"
Id Example Poll of Ours Poll of Photo2ClipArt Poll of Equivalent
1 Can 29 3 0
2 Cone 26 4 2
3 Purple-circle 22 10 0
4 Hammer 16 10 6
5 Tiger 24 5 3
6 Cherry 17 11 4
7 Torch 17 9 6
8 Rocket 23 8 1
9 Plane 22 8 2
10 Soda 21 7 4
11 Trees 13 19 0
12 Teapot1 26 4 2
13 Syn0 10 6 16
14 Syn2 3 14 15
15 Syn4 13 4 15
16 Battery 26 6 0
17 Car 17 7 8
18 Coffee 21 7 4
19 Cow 26 3 3
20 Truck 24 3 5
21 House 23 7 2
22 Sound 27 4 1
23 Bear 21 6 5
24 Mouse 16 12 4
25 Lamp 19 3 10
26 Teapot2 21 6 5
27 Shoe1 27 2 3
28 Syn6 21 6 5
29 Syn8 16 4 12
30 Syn1 13 7 12
31 Syn6 13 3 16
32 Syn7 16 2 14
Average Voting Rate 629(61.4%) 210(20.5%) 185(18.1%)
Conclusion In 30 examples (94%),   our results get more votes than Photo2ClipArt.
Table 3. Voting results of "Q3 (editing convenience): Which decomposition is more convenient for editing?"
Id Example Poll of Ours Poll of Photo2ClipArt Poll of Equivalent
1 Can 29 3 0
2 Cone 30 1 1
3 Purple-circle 19 9 4
4 Hammer 19 10 3
5 Tiger 20 8 4
6 Cherry 18 9 5
7 Torch 15 8 9
8 Rocket 20 9 3
9 Plane 21 9 2
10 Soda 20 7 5
11 Trees 18 12 2
12 Teapot1 24 4 4
13 Syn0 8 4 20
14 Syn2 2 12 18
15 Syn4 13 3 16
16 Battery 30 2 0
17 Car 16 7 9
18 Coffee 21 9 2
19 Cow 25 6 1
20 Truck 23 6 3
21 House 25 6 1
22 Sound 26 4 2
23 Bear 21 9 2
24 Mouse 14 14 4
25 Lamp 18 5 9
26 Teapot2 22 7 3
27 Shoe1 26 3 3
28 Syn6 23 2 7
29 Syn8 15 2 15
30 Syn1 10 6 16
31 Syn6 10 5 17
32 Syn7 13 6 13
Average Voting Rate 614(60.0%) 207(20.0%) 203(20.0%)
Conclusion In 30 examples   (94%), our results get more votes than Photo2ClipArt.